- Move all imports to top level for immediate dependency checking
- Add environment validation to ensure py312 conda environment is active
- Remove lazy import strategy that was causing dependency issues
- Ensure consistent environment setup across development workflow

- Change read_full_search_pickles to read_full_search_pickle (singular)
- Ensure consistency between config.py field names and run.py usage
- Resolves KeyError when accessing configuration fields
- Configuration system now fully functional

- Extract Stage 1 (initial search) logic into run_initial_search() function
- Add proper parameter passing to make function self-contained
- Return all necessary variables from initial search for use in main workflow
- Improve code organization and maintainability
- Function signature: run_initial_search(config, args_dict, result_dir, result_temp_results_initial_search, pickle_config)
- Returns: (df_fragment, df_psms, df_fragment_max, df_fragment_max_peptide, dlc_transfer_learn, config)

- Extract Stage 2 (targeted search with retention time partitioning) into run_targeted_search() function
- Add proper parameter passing for all dependencies including DataFrames from Stage 1
- Return all necessary variables for use in subsequent workflow stages
- Improve code modularity and maintainability
- Function signature: run_targeted_search(config, args_dict, result_dir, pickle_config, df_fragment, df_psms, df_fragment_max, df_fragment_max_peptide, dlc_transfer_learn)
- Returns: (df_fragment, df_psms, df_fragment_max, df_fragment_max_peptide, dlc_transfer_learn, config)
- Maintains full search pickle logic and DeepLC model training functionality

- Set all pickle read flags to false in config.json to force fresh computation
- Change default plot output directory from 'xics' to 'results' in plotting utility
- Ensure consistent behavior for development and testing workflow

- Delete empty test files: test_io_utils.py, test_pickling_simple.py
- Remove empty workflow generation files: WorkflowDiagramGenerator.java, generate_diagrams.sh, generate_workflow_figure.py
- Clean up repository by removing files with no content
- These were likely leftover from development/refactoring process

- Add new simplified flat configuration system (config_new.py)
  * Single flat parameter structure instead of nested sections
  * Override mechanism using suffixes (_initial_search, _full_search)
  * Automatic default values for all parameters
  * Smart parameter resolution with fallback to base values

- Implement automatic backwards compatibility
  * Auto-detects old nested format vs new flat format
  * Converts legacy configs transparently without user intervention
  * Maintains full functionality for existing config.json files
  * Fixes 'unexpected keyword argument sage_basic' error

- Update workflow integration (run.py)
  * Replace complex config parsing with simplified system
  * Add command line argument support for config file path
  * Update all function signatures to use new config object
  * Maintain compatibility with existing workflow code via to_legacy_format()

- Add configuration examples and tests
  * config_simple.json: Example of new flat format with overrides
  * test_config_compatibility.py: Comprehensive backwards compatibility test
  * Verify both old and new formats work correctly

- Key benefits:
  * Dramatically reduced complexity (from ~100 lines parsing to simple flat structure)
  * User-friendly override mechanism for initial vs full search parameters
  * Full backwards compatibility - no migration required
  * Self-documenting with comment support in JSON
  * Type-safe with proper dataclass structure

This resolves the configuration complexity issues while maintaining
full backwards compatibility for existing users.

- Rename config_new.py → config.py (new simplified system becomes primary)
- Backup old config system as config_old.py for reference
- Update all import statements:
  * run.py: from config_new import → from config import
  * test_config_compatibility.py: updated import
  * demo_new_config.py: updated import (also added to git)

- Benefits of replacement:
  * Clean primary config.py with simplified flat structure
  * Full backwards compatibility maintained
  * All existing functionality preserved
  * Users can now import from 'config' instead of 'config_new'
  * Old complex system preserved as config_old.py for reference

All tests pass:
✅ New config.py works correctly
✅ Backwards compatibility maintained
✅ Demo script works
✅ Compatibility test passes
✅ Main workflow accepts both config formats

The new simplified config system is now the primary configuration system.